In [1]:
# import all packages and set plots to be embedded inline
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sb
%matplotlib inline
Load in your dataset and describe its properties through the questions below. Try and motivate your exploration goals through this section.
In [2]:
# load data
df = pd.read_csv('data.csv')
df.head()
Out[2]:
In [3]:
df.describe()
Out[3]:
In [4]:
#changing data type of start/end_time to datetime and bike_share_for_all_trip to bool.
df.start_time = pd.to_datetime(df.start_time)
df.end_time = pd.to_datetime(df.end_time)
df.bike_share_for_all_trip = (df.bike_share_for_all_trip == 'Yes')
In [5]:
df.info()
In [6]:
df.isnull().sum()
Out[6]:
The dataset have 183412 record so it's better to drop all null values¶
In [7]:
df.dropna(inplace=True)
df.isnull().sum()
Out[7]:
In [8]:
print(df.shape)
What is the structure of your dataset?¶
The dataset consists of 183412 record and 16 columns. Out of these 16 columnsm nine are numerical, two are datetime, four are object and one is boolean type. The columns can be grouped into four main categories
- trip related
- duration_sec
- start_time/end_time
- station replated
- start_station_id/end_station_id
- start_station_name/end_station_name
- start/end_station_latitude/longitude
- bike replated:
- bike_id
- bike_share_for_all_trip
- user related:
- user_type: whether the user is customer or subsriber
- member_birth_year
- member_gender
What is/are the main feature(s) of interest in your dataset?¶
I would like to invistigate which factors affect trip duration
What features in the dataset do you think will help support your investigation into your feature(s) of interest?¶
- duration_sec
- start_time/end_time
- start/end_station_name
- start/end_station_latitude/longitude
- user_type
- member_birth_year
- member_gender
Univariate Exploration¶
In this section, investigate distributions of individual variables. If you see unusual points or outliers, take a deeper look to clean things up and prepare yourself to look at relationships between variables.
In [9]:
## Here I will create two columns that will help me doing invistigation
## add two columns for days and hours
df['start_time_weekday'] = df['start_time'].dt.day_name()
df['start_time_weekday'] = pd.Categorical(df.start_time_weekday, ordered=True,
categories=['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday'])
df['start_time_hour'] = df['start_time'].dt.hour
df.head()
Out[9]:
In [10]:
def get_bins(column, max_value:int=None, min_value:int=None, number_of_bins:int=20):
"""
split data range into required number of bins
Args:
column: pandas series
max/min_value: max and min value for columns
number_of_bins: required number of bins, default = 20
return: numpy array have the splitted numbers
"""
if not max_value:
max_value = column.max()
if not min_value:
min_value = column.min()
step = (max_value - min_value) / number_of_bins
bins = np.arange(min_value, max_value+step, step)
return bins
def get_column_key_value(column):
"""
takes a panda series and return a tuple that have
the series keys and values
"""
return column.value_counts().keys(), column.value_counts()
Let's statr by invistigating the main variable "duraion_sec"¶
In [11]:
# plotting the histogram of trip duration
plt.hist(data=df, x='duration_sec', bins=get_bins(df.duration_sec))
print(f"Max Duration = {df.duration_sec.max()} ms")
The data is concentrated in the lower part. It's probably better to limit the data at 5k and convert into log scale¶
In [12]:
# plotting trip duraion in log scale
bins = get_bins(df.duration_sec, max_value=5e3, number_of_bins=1000)
plt.hist(data = df, x = 'duration_sec', bins = bins)
plt.xscale('log')
plt.xticks([100, 200, 500, 1e3, 2e3, 5e3], [100, 200, 500, '1k', '2k', '5k']);
Most data are below 2k with a peak around 400, The analysis should focus in this portion¶
In [13]:
df = df[df.duration_sec < 2e3]
Now, let's look about users birth year, but it would be better to see the age instead of birth year, so first let's extract age from this column and run the analysis, the data is collected in 2019, therefore subtracting year from 2019 would fetch the required info¶
In [14]:
df['age'] = np.array(2019 - df.member_birth_year).astype(int)
In [15]:
plt.hist(df.age, bins=get_bins(df.age, number_of_bins=10));
Most values of age is less than 70 with peak around 30¶
In [16]:
df = df[df.age < 70]
In [17]:
plt.hist(df.age, bins=get_bins(df.age));
Let's get a look on station start and end points¶
In [18]:
df.columns
Out[18]:
In [19]:
df.start_station_id.hist(bins=100);
In [20]:
df.end_station_id.hist(bins=100);
Same stations are frequent as start and end stations¶
let's look at memeber type and gender¶
In [21]:
# bar plot for user type
x, y = get_column_key_value(df.user_type)
sb.barplot(x=x, y=y);
In [22]:
# bar plot for member gender
x, y = get_column_key_value(df.member_gender)
sb.barplot(x=x, y=y);
Subscriber and Male are most probable¶
Which hour/day have the most trip count?¶
In [23]:
# Plot bar plot for starting time vs days
x, y = get_column_key_value(df.start_time_weekday)
sb.barplot(x=x, y=y);
plt.xticks(rotation = 45);
As expected, The least days are the weekend days¶
In [24]:
# Plot bar plot for starting time vs hours
x, y = get_column_key_value(df.start_time.dt.hour)
sb.barplot(x=x, y=y);
Again as expected, there are 2 peaks at 8 and 17, "the start and the end hours of work day".¶
Discuss the distribution(s) of your variable(s) of interest. Were there any unusual points? Did you need to perform any transformations?¶
The trip duration has a huge amount of values, most of them are in the lower part. so I used log scale to generate a better visualization and remove outliers
Of the features you investigated, were there any unusual distributions? Did you perform any operations on the data to tidy, adjust, or change the form of the data? If so, why did you do this?¶
I subtracted 2019 from the member_birth_year to get Age
Bivariate Exploration¶
In this section, investigate relationships between pairs of variables in your data. Make sure the variables that you cover here have been introduced in some fashion in the previous section (univariate exploration).
First, let's look at duration vs age¶
In [25]:
# plot scatter plot and heat map for age and trip duration
# scatter plot
plt.scatter(df.age, df.duration_sec, alpha = 0.2, marker='.');
plt.title('Scatter plot for Trip duration vs Age')
plt.xlabel('Age')
plt.ylabel('Trip Duration (sec)')
plt.show()
# Heat map
# Specify bin edges
bins_x = get_bins(df.age)
bins_y = get_bins(df.duration_sec)
plt.hist2d(data=df, x='age', y='duration_sec', bins=[bins_x, bins_y]);
plt.colorbar();
plt.title('Heatmap for Trip duration vs Age')
plt.xlabel('Age')
plt.ylabel('Trip Duration (sec)')
plt.show()
Both short and long trips are most frequently made by bikers between 25 and 35¶
Let's explre the duration member vs member gender¶
In [26]:
sb.displot(data=df, x='duration_sec', hue='member_gender', bins=get_bins(df.duration_sec));
In [27]:
## violin and box plot for member gender vs trip duration in sec
sb.violinplot(data=df, x='member_gender', y='duration_sec');
plt.show()
sb.boxplot(data=df, x='member_gender', y='duration_sec');
Let's explre the duration member vs uesr_type¶
In [28]:
## violin and box plot for user type vs trip duration in sec
sb.violinplot(data=df, x='user_type', y='duration_sec');
plt.show()
sb.boxplot(data=df, x='user_type', y='duration_sec');
We can see that on average that customers do longer trips than subscribers¶
In [29]:
# Creating Facet Grid
# add histogram of trip duration agains days
g = sb.FacetGrid(df, col='start_time_weekday', col_wrap=4);
g.map(sb.histplot, 'duration_sec');
Most long trips are done on weekends¶
In [30]:
# Creating Facet Grid and add histogram of trip duration agains hours
g = sb.FacetGrid(df, col='start_time_hour', col_wrap=6)
g.map(sb.histplot, 'duration_sec');
Most long trips are done between 11 and 17, There are nearly no long trips done between 0 and 6¶
In [31]:
## plot the summation of trips against start station id
df.groupby('start_station_id').sum().reset_index().plot(x='start_station_id', y='duration_sec');
In [32]:
## plot the summation of trips against end station id
df.groupby('end_station_id').sum().reset_index().plot(x='end_station_id', y='duration_sec');
The same stations are associated with longer/shorter trip whether as start or end destination¶
Talk about some of the relationships you observed in this part of the investigation. How did the feature(s) of interest vary with other features in the dataset?¶
The trip duration is strongly associated with age, start/end station, and user_type, and of course the trip hour
Did you observe any interesting relationships between the other features (not the main feature(s) of interest)?¶
Intrestingly, Females have higher trip duration than males. Weekends also have higher trip duration, maybe because users ride bike for fun instead of going to a pre-determined place.
Multivariate Exploration¶
Create plots of three or more variables to investigate your data even further. Make sure that your investigations are justified, and follow from your work in the previous sections.
Here I will invistigate the relationship between trip_duration, age and [user_type, member_gender, start_time_week/hour]¶
In [33]:
## Create subplots consisting of 1 row and 2 columns
fig, ax = plt.subplots(1, 2)
## set axis 0 as the current axis
plt.sca(ax[0])
## Point plot for trip duration vs member gender
sb.pointplot(data = df, x = 'member_gender', y = 'duration_sec', hue = 'user_type', dodge = 0.3, linestyles = "");
## set axis 1 as the current axis
plt.sca(ax[1])
## Bar plot for trip duration vs member gender
sb.barplot(data = df, x = 'member_gender', y = 'duration_sec', hue = 'user_type');
ax[1].legend();
Customers do longer trips than subscribers, Females do longer trips than males and other¶
In [34]:
## Facet Grid for trip duration vs Age splitted by gender
g = sb.FacetGrid(data=df, col='member_gender', col_wrap=2, height=5);
g.map(plt.scatter, 'age', 'duration_sec', alpha=0.01);
Older males do longer trips than females¶
In [35]:
## Facet Grid for trip duration vs Age splitted by days
g = sb.FacetGrid(data=df, col='start_time_weekday', col_wrap=3, height=5);
g.map(plt.scatter, 'age', 'duration_sec', alpha=0.01);
It seems those who are at working age are doing most of the trips¶
In [36]:
## Facet Grid for trip duration vs Age splitted by hours
g = sb.FacetGrid(data=df, col='start_time_hour', col_wrap=4, height=5);
g.map(plt.scatter, 'age', 'duration_sec', alpha=0.1);
Older people do bike trips at the morning more than any othet time¶
Talk about some of the relationships you observed in this part of the investigation. Were there features that strengthened each other in terms of looking at your feature(s) of interest?¶
We observed that customer do longer trips than subscribers, Although females do longer trips than males, older males do longer trips than females. The most portion using go ford bikes are those who at the working age, and they probably do their trips on weekdays
Were there any interesting or surprising interactions between features?¶
It is nice to see older people do morning bike rides.
At the end of your report, make sure that you export the notebook as an html file from the
File > Download as... > HTMLmenu. Make sure you keep track of where the exported file goes, so you can put it in the same folder as this notebook for project submission. Also, make sure you remove all of the quote-formatted guide notes like this one before you finish your report!
In [ ]: